edam 

'f 

Educational Sciences: Theory & Practice - 13(3] • 1638-1644 
®2013 Educational Consultancy and Research Center 
www.edam.com.tr/estp 
DOI: 10.12738/estp.2013.3.1692 

The Effectiveness Level of Material Use in Classroom 
Instruction: A Meta-analysis Study 

Zeynel KABLAN 3 Beyda TO PAN b 

Kocaeli University Kocaeli University 

Burak ERKAN C 

Kocaeli University 


Abstract 

In this study, the aim was to combine the results obtained in independent studies aiming to determine the effec¬ 
tiveness of material use. The main question of the study is: "Does material use in classroom instruction improve 
students’ academic achievements?” To answer this question, the meta-analysis method was employed. Meta¬ 
analysis is the method employed in order to statistically analyze the quantitative data collected in independent 
and multiple studies carried out on similar topics, and to reach a general judgment regarding the results of 
these studies. Certain criteria were used in order to decide which researches would be included in the meta¬ 
analysis. Based on these criteria, it was decided to include 57 experimental studies in the meta-analysis. As a 
result of this analysis; it was determined that the material use in classroom instruction has a positive effect in 
terms of academic achievement, and that this effectiveness level does not differ with respect to stages of educa¬ 
tion, course types and material types. 
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Education programmes that have been applied since 
2004-2005 school year in Turkey are focused on stu¬ 
dent-centred and active instruction instead of teach¬ 
er-centred instruction (Arslan & Ozpinar, 2009; Peker 
& Halat, 2008). However, most teachers state that class¬ 
room applications of the new programmes are in trou¬ 
ble because of lack of structure and materials (Ak<;a, 
2007; Akkaya, 2008; Duru & Korkmaz, 2010; Kalender, 
2006; Me§in, 2008; Peker & Halat, 2008; Saglik, 2007; 
§ahin, 2010). Effective use of materials is considered 
an important aspect of new educational programmes. 


Research on educational materials listed sever¬ 
al benefits for students, such as making learning 
easier, supporting active and individual learning, 
providing real life experiences, improving critical 
thinking, problem solving and creative thinking 
skills (Ak<;ay, Tiiysuz, Feyzioglu, & Oguz, 2008; Ap- 
person, Laws, & Scepansky, 2006; Demirel, Seferog- 
lu, & Yagci, 2001; i§man, 2005; Knapp & Glenn, 
1996; Ko§ar et al., 2003; Lowry, 1999; Riza, 2000). 
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This study aimed to compile the results of indepen¬ 
dent research on effectiveness of using educational 
materials. With the increasing number of research 
and body of knowledge, it might be difficult to 
reach targeted information. Thus, a detailed re¬ 
search approach is needed to bring related knowl¬ 
edge together, analyse it and lead way to future 
studies. Meta-analysis method aims to join results 
of individual studies conducted in different times 
and places. Since this method provides an increased 
sample size, researchers are able to use the statisti¬ 
cal results of different studies to reach more precise 
information (Yildiz, 2002). 

In this meta-analysis study, the basic question is 
“whether the use of educational materials affect 
students’ achievement or not?” The effectiveness of 
educational materials based on the type of material 
that is used, educational level and subject matter 
were also examined. 

Method 

Meta-analysis is a research method that combines 
and analyses statistical information from different 
studies on a specific topic and tries to identify pat¬ 
terns among study results (Glass, 1976; Johnson, 
Johnson, & Stanne, 2000; Lipsey & Wilson 2001; 
Saglam & Yiiksel, 2007). 

In general, meta-analysis method has three stages: 

1) Selection of related studies. 

2) Coding the data and calculating effect sizes. 

3) Statistical analysis of effect sizes and interpreta¬ 
tion of results (Hoffler & Leutner, 2007). 

Data Collection 

The criteria for selecting the studies to be included 
in the meta-analysis are listed below (Camnalbur & 
Erdogan, 2008; Okursoy, 2009; Ozcan & Bakioglu, 
2010; Springer, Stanne, & Donovan, 1999; §ahin, 
2005; Top<;u, 2009): 

1. Studies that investigated the effectiveness of 
educational materials conducted in Turkey and 
published between the years 2000 and 2012 were 
analysed. 

2. Quasi-experimental and experimental studies 
that used educational materials in the experi¬ 
ment groups were included in this meta-analysis. 
Studies on distance learning, computer-based 
learning and studies that used worksheets were 
not included in the analysis. 


3. Studies that did not report statistical informa¬ 
tion necessary to calculate effect sizes were not 
included in the analysis. 

4. For studies that used more than one experiment 
group, data from the group that used educational 
materials were included in the analysis. 

5. For studies that used more than one achievement 
test, data for one of the achievement tests were 
randomly selected and included in the analysis. 

To get the sample of the study, academic articles 
that were published in national journals, theses and 
dissertations that were listed in the YOK national 
thesis/dissertation centre, and conference papers 
that were published in proceedings or conference 
books were collected. Based on the above criteria, 
57 studies (appendix 1) that investigated the ef¬ 
fectiveness of educational materials in academic 
achievement were included in the analysis. 

Data Coding 

Data for this study were coded in two parts. First, 
summary of the information on each study were 
coded under eight categories; then, statistical infor¬ 
mation necessary to conduct meta-analysis, such as 
sample sizes, means 

and standard deviations were extracted. 

Dependent Variables 

Dependent variables in this study were the effect 
sizes calculated based on the achievement scores in 
each study. Effect size is described as a standard val¬ 
ue for studies that use different measurement tools 
(Bernard et al., 2004). 

Discrete Variables 

There were three discrete variables based on the 
characteristics of the studies. In order to assess the 
relationships between the effects and various study 
characteristics, following discrete variables were 
coded (Camnalbur & Erdogan, 2008; Tarim, 2003): 

- Educational levels that were examined in 57 stud¬ 
ies were grouped as primary, secondary and univer¬ 
sity level. 

- Subject matters that were examined were Maths, 
Science and Technology, Social Studies, Sciences 
(Physics, Chemistry, Biology), Social Sciences (His¬ 
tory, Geography), Turkish, English, Computer, Mu¬ 
sic and Technical Art. 
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- Types of educational materials used in studies 
included computer presentations, hands-on mate¬ 
rials, cartoons, concept maps, and combination of 
different materials. In the studies, computer pre¬ 
sentation was in either PowerPoint or animation 
format. Hands-on materials are described as mate¬ 
rials that can be touched and moved around easily 
by students (Hynes, 1986). Cartoons are used for 
specific content and reveal important information 
about it (Oz§ahin, 2009). The concept maps are two 
dimensional and detailed diagrams that show rela¬ 
tionships among concepts (Kaptan, 1999). A com¬ 
bination category was also identified for this study 
when more than one material are used. 

Data Analysis 

In data analysis, treatment effectiveness meta-anal¬ 
ysis method is used. In this method, the purpose 
is to calculate the difference between the means of 
experiment and control groups with the formula 
d=(Xe-Xc)/SD (Hunter & Schmidt, 1990). In order 
to interpret findings of different studies, a common 
measure of effect size is calculated. For calculating 
the effect sizes in this study, “Hedges g” and confi¬ 
dence intervals of %95 were used. For interpreting 
effect sizes, Cohens (1992) guidelines were used. 


Results 

In terms of education level, most of the studies were 
conducted at the primary level (61.4%), followed by 
secondary (28.1%) and university level (10.5%). In 
terms of subject, 16 studies involved Science and 
Technology, 11 of them involved Sciences, 8 So¬ 
cial Sciences, 7 Social Studies, 7 Maths, 3 English, 
1 Computer, 1 Turkish, 1 Music, and 1 Technical 
Art. In terms of materials, computer presentations 
(29.8%) was the most frequently used material type, 
followed by concept maps (22.8%), combination 
materials (17.6%), cartoons (15.8%) and hands-on 
materials (14%). 

According to the random effects model, the data 
from 57 studies yielded the standard error of 0.106, 
95% confidence intervals of 1.4846 and 1.0581, and 
the effect size of 1.269. The magnitude of the effect 
size is large according to Cohens (1992) classifica¬ 
tion. It can be concluded that the use of educational 
materials in classroom instruction has a strong pos¬ 
itive contribution to academic success. 


Effectiveness of Educational Materials by Level 

As a result of the homogeneity test, the Q-statistic 
is calculated as 0.388. According to chi-square table 
with 2 degrees of freedom and confidence intervals 
of 95%, the critical value is considered to be about 
5.991. In this study, since the Q-statistic (0.388) is 
smaller than the critical value of 5.991, the hypoth¬ 
esis of homogeneity of the distribution of effect 
sizes has been accepted according to fixed effects 
model. In other words, the distribution is homo¬ 
geneous and there are no significant differences in 
effect sizes (QB=0.388; p=0.824) among different 
educational levels. 

Effectiveness of Educational Materials by Subject 
Matter 

As a result of the homogeneity test, the Q-statistic 
is calculated as 4.759. According to chi-square table 
with 4 degrees of freedom and confidence inter¬ 
vals of 95%, the critical value is considered to be 
about 9.490. In this study, since Q-statistic (4.759) 
is smaller than the critical value of 9.490, the hy¬ 
pothesis of homogeneity of the distribution of effect 
sizes has been accepted according to fixed effects 
model. In other words, the distribution is homo¬ 
geneous and there are no significant differences in 
effect sizes (QB=4.759; p= 0.446) among different 
subject matters. 

Effectiveness of Educational Materials by the 
Type of Material 

As a result of the homogeneity test, the Q-statistic 
is calculated as 4.106. According to chi-square table 
with 4 degrees of freedom and confidence intervals 
of 95%, the critical value is considered to be about 
9.490. In this study, since the Q-statistic (4.106) is 
smaller than the critical value of 9.490, the hypoth¬ 
esis of homogeneity of the distribution of effect 
sizes has been accepted according to fixed effects 
model. In other words, the distribution is homo¬ 
geneous and there are no significant difference in 
effect sizes (QB=4.106; p=0.392 ) among different 
types of materials. 
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